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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
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Status 

1 )M Responsive to communication(s) filed on 22 August 2007 . 
2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-21,32-38 and 41-50 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) K Claim(s) 1-21,32-38 and 41-50 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 30 March 2004 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 
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application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Election/Restrictions 

1. Applicant's election without traverse of species I in the reply filed on August 
22, 2007 is acknowledged. 

2. Claims 22-31, 39 and 40 are withdrawn from further consideration pursuant 
to 37 CFR 1.142(b) as being drawn to a nonelected species, there being no allowable 
generic or linking claim. Election was made without traverse in the reply filed on 
8/22/2007. 

3. A new grouping of claims for the Restriction Requirement is supplied hereby 
to correct the group of claims corresponding to their appropriate species. 



4. This application contains claims directed to the following patentably distinct 
species: 

a. Species I, figs. 1*3 and 6-10, directed towards processing and outputting 
multimedia data in a variety of formats, as per claims 1-21, 32-38 and 41-50; 

b. Species II, figs. 4, 5 and 8, directed towards detecting events in 
multimedia data and performing an action in response to the event, then processing 
the multimedia data by inserting a multimedia object into the multimedia 
document, as per claims 22*31, 39 and 40. 
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5. The species are independent or distinct because they are shown to be 
separately usable. In the instant case, Species II has a separate utility such as 
performing an action when detecting an event in the multimedia data; comparing a 
profile of the event; and outputting a waveform representing the multimedia data 

1. Applicant is required under 35 U.S.C. 121 to elect a single disclosed species 
for prosecution on the merits to which the claims shall be restricted if no generic 
claim is finally held to be allowable. Currently, claim 1 is generic. 

2. Applicant is advised that a reply to this requirement must include an 
identification of the species that is elected consonant with this requirement, and a 
listing of all claims readable thereon, including any claims subsequently added. An 
argument that a claim is allowable or that all claims are generic is considered 
nonresponsive unless accompanied by an election. 

3. Upon the allowance of a generic claim, applicant will be entitled to 
consideration of claims to additional species which depend from or otherwise 
require all the limitations of an allowable generic claim as provided by 37 CFR 
1.141. If claims are added after the election, applicant must indicate which are 
readable upon the elected species. MPEP § 809.02(a). 

4. Applicant is reminded that upon the cancellation of claims to a non-elected 
invention, the inventorship must be amended in compliance with 37 CFR 1.48(b) if 
one or more of the currently named inventors is no longer an inventor of at least one 
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claim remaining in the application. Any amendment of inventorship must be 
accompanied by a request under 37 CFR 1.48(b) and by the fee required under 37 
CFR 1.17(0. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 

that form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

6. Claims 1-3, 5-13, 15, 16, 18, 32-36, 38, 41-47, 49 and 50 are rejected under 35 
U.S.C. 102(b) as being anticipated by Tonomura etal (US 5, 576,950). 

Claim l: 

Tonomura discloses a method for receiving multimedia data by a multimedia 
processing device capable of outputting a document (abstract; col. 2, lines 43-54; col. 

4, lines 53-67; col. 5, lines 1-21); the device configured to process the multimedia 
data and output the processed multimedia data (col. 4, lines 17-33, 40-44, 53*67; col. 

5, lines 1-21; col. 11, lines 56-67; col. 12, lines 1-52). 
Claim 2: 

Tonomura teaches multimedia data is video and audio data (col. 4, lines 53- 

64). 
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Claim 3: 

Tonomura teaches the video signal source is a multimedia data stream (col. 
16, lines 57-59). 
Claim 5: 

Tonomura teaches receiving multimedia data from a storage device (col. 1, 
lines 6-13; col. 3, lines 1-10; col. 4, lines 7-52). 
Claims 6, 7, 35, 49 and 5Q: 

Tonomura teaches identifying a pre-determined multimedia event in the 
multimedia data (col. 2, lines 43-46, 63-67; col. 3, lines 1-10; col. 11, lines 56-67), 
and performing an action if the event is identified (col. 12, lines 14-23, 37-52). 
Claims 8 and 36: 

Tonomura teaches performing an action associated with a multimedia event 
in an event table when the event is identified (col. 2, lines 63-67; col. 3, lines 1-10). 
Claim 9: 

Tonomura teaches a system that creates graphical representations of video 
and audio signals. Tonomura explains that audio and video signals are grabbed as 
digital image data (col. 3, lines 1-28; col. 5, lines 55-67; col. 6, lines 1-12). 
Claims 10 and 47: 

See claim 1. Tonomura teaches outputting the processed multimedia data by 
writing the processed multimedia data to an archive file (col. 2, lines 63*67; col. 3, 
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lines 1-10; col. 4, lines 6-20, 40-52 col. 6, lines 3-22; p. 12, lines 1-15, 41-52; fig. 1). 
Claim 11: 

Tonomura teaches that processed multimedia data are a representation of 
the received multimedia data (col. 11, lines 56-67; col. 12, lines 15-23). 
Claim 12: 

Tonomura teaches outputting the processed multimedia data by outputting 
portions of the multimedia data as video paper (col. 6, lines 3-21; col. 12, lines 14- 
23). 

Claim 13: 

Tonomura teaches outputting the processed multimedia data by outputting 
portions of the multimedia data as a paper document (col. 2, lines 30-40, 43-54; col. 
4, lines 30-33; col. 5, lines 1-30, 49-52; col. 11, lines 26-32, 50-67; col. 12, lines 29- 
32). 

Claim 15: 

See claim 1. Tonomura teaches capturing a video frame from video data and 
saving it to a file for later access (col. 6, lines 3-12; col. 11, lines 56-67). 
Claims 16. 34 and 38: 

Tonomura teaches outputting comprises saving the processed multimedia 
data to a storage medium and indexing the processed data (col. 2, lines 30-54; col. 5, 
lines 57-67; col. 10, lines 50-53; col. 11, lines 56-63; col. 12, lines 15-23). 
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Claim 18: 

See claim 1. Tonomura teaches a capturing device for capturing multimedia 
data (col. 4, lines 14-29; col. 5, lines 1-43; col. 6, lines 52-67; col. 11, lines 55-67; col. 
12, lines 29-52), and generating a control signal to be transmitted to the peripheral 
device (col. 4, lines 53-67). 
Claims 32. 33. 44 and 45: 

See claims 1, 12 and 13. Tonomura teaches outputting processed multimedia 
data in paper-based (col. 2, lines 30-40, 43-54; col. 4, lines 30-33; col. 5, lines 1-30, 
49-52; col. 11, lines 26-32, 50-67; col. 12, lines 29-32) and electronic formats (col. 6, 
lines 3-21; col. 12, lines 14-23). 
Claims 41, 42 and 43: 

See claim 1. Tonomura teaches a printing device having a printer, a 
processor for processing multimedia data, a memory for storing processed 
multimedia data, and an output system for outputting the multimedia data (figs. 1 
and 13 A). 
Claim 46: 

Tonomura teaches an indexing/mapping module for mapping contents of the 
multimedia data to a file (figs. 1, 3, 4, 6, 11). 
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Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

8. Claims 4, 14, 17 and 37 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tonomura and Behzad Shahraray et al (Shahraray hereinafter! 
article entitled "Pictorial Transcripts* Multimedia Processing Applied to Digital 
Library Creation"). 

Claim 4: 

Tonomura does not teach that multimedia data is received from a network. 
However, Shahraray discloses a system for retrieving multimedia data from the 
Internet, creating representations of information contained in the multimedia data, 
storing the representations for later access, and outputting the information in 
electronic form (pages 581-586). Therefore, it would have been obvious to a person 
having ordinary skill in the art at the time of invention to modify Tonomura's 
method of retrieving multimedia information to include Shahraray's teaching of 
retrieving multimedia data from the Internet because it enables users to search, 
browse and retrieve media data from a vast collection of information around the 
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world. 

Claims 14 and 37: 

See claims 1 and 4. Tonomura/Shahraray teaches storing multimedia data 
for later access over the Internet (server/client), (Shahraray: page 581). 
Claim 17: 

See claims 1 and 4. Tonomura does not teach transcribing audio data into 
text and outputting the text. However, Shahraray teaches retrieval of video and 
audio data, processing and outputting the media data in text and pictorial form 
(Shahraray: pages 584*586). Thus, it would have further obvious to an artisan in 
the art at the time of invention to modify Tonomura's method of capturing and 
outputting multimedia data to include Shahraray' s teaching of converting and 
outputting audio data into text data to provide a media representation because 
users are enabled to generate a summary of multimedia data for easy and fast 
review of data that is of interest to the user. 

9. Claims 19-21 and 48 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tonomura and Strub et al (US 2004/0156616 Al). 

Claims 19. 20, 21 and 48: 

See claim 18. Tonomura teaches processing multimedia data for localization 
(col. 5, lines 1-22; col. 7, lines 35-51; col. 9, lines 24-25; col. 11, lines 50-66). 
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Tonomura does not teach a control signal for orienting the peripheral device to 
improve monitoring quality; performing audio or video localization; and controlling 
orientation of a microphone or video capture device by using the control signal. 
However, Strub discloses a system and method for using audio and video recording 
devices to record an event (abstract; p. 1, par. 0003; p. 2, par. 0015-0016). Strub 
teaches that an audio and video communication device can include a transmitter for 
transmitting a signal representing communicative information to be displayed or 
processed (p. 5, par. 0036). Strub explains that the system includes a position 
sensing device for ascertaining the position (location and/or orientation) of the 
recording unit at any point in time (p. 6, par. 0039; p. 9, par. 0064). Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the 
invention was made to modify Tonomura's invention to include Strub's teaching of 
using audio and video localization to control orientation of the peripheral devices 
because it enables capture of quality video and audio obtained at a distance without 
much user interaction. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

11. Any inquiry concerning this communication or earlier communications from 
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the examiner should be directed to X. L. Bautista whose telephone number is (571) 
272*4132. The examiner can normally be reached on Monday-Thursday 8:00AM- 
6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Weilun Lo can be reached on (571) 272-4847. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 



12. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 
272-1000. r> n 



273-8300. 




X. L. BAUTISTA 
PRIMARY EXAMINER 



xlb 

August 30, 2007 



